Spatial filling methods, as illustrated in FIG. 2B try to guess the disoccluded texture according to various spatial correlation assumptions. Background textures are used to fill the disoccluded regions 110, as those regions normally hide in the reference view and contain further depth. One prior solution filled the disocclusions 110 with an average of the available background pixels in their neighborhood. However, the filling results tend to be over-smoothed when there are complex textures in the background. To overcome this over-smoothed artifact, classical spatial filling method such as exemplar based inpainting can be used. However, this approach performs worse in hole filling than in general images, because 1) the neighborhoods of the inpainting area are on the disparity boundaries, containing a lot of ambiguities, and 2) the texture to be inpainted is not constrained in the background. Other prior solutions extended traditional exemplar-based inpainting to depth-exemplar-based inpainting, where the depth is used to enforce the structure diffusion apart from texture diffusion. However, this approach is still far from real-time processing. In addition, the image quality of spatial domain filling methods depend on their spatial correlation assumptions.
Temporal domain filling methods may be more useful than special domain filling, considering it is able to reveal the ground-truth texture pattern in the disoccluded regions 110 by exploring more frames, while the spatial domain filling methods merely guess. FIG. 2C illustrates methods for filling using temporal past frames 206, and FIG. 2D illustrates methods for filing using both temporal past frames 206 and temporal future frames 208. One temporal solution used a running average background model in temporal domain by firstly segmenting background from the scene and then dynamically updating it. However, this approach may not work with slanted surfaces or non quasi-static background scenes. Another prior method filled the holes 204 by the background extracted from median filtering of the frames in the background layer. However, suitable background information is limited in the neighboring temporal segment. It was also an off-line process with relatively high memory occupation.